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IR camera 
Technical Field 

The present invention relates to an infrared (IR) camera as defined in the preamble 
of claim 1 . 

Background 

It is of great interest, for example, to be able to process images taken by infrared 
(IR) cameras. IR cameras are used to capture images showing the temperature of 
different objects, or parts of objects, in a view. Such thermal images are used, for 
example, when searching for defects in devices, since defect parts of devices are 
often heated before they break. 

An IR camera typically captures information of temperatures from about -60 de- 
grees Celsius to about 2000 degrees Celsius. This range is called the thermal camera 
dynamic range. Typically, thermal images will show the background ambient tem- 
perature and some hot or cold objects that will be subject of the inspection. Back- 
ground ambient temperature will vary from outdoor winter temperature to a hot 
summer day temperature. The inspected object (a thermal anomaly) differs from the 
background as a hotter or cooler object within or outside the typical background 
range. Thus, the thermal images will comprise a much smaller range than the ther- 
mal camera dynamic range. The images, or sequence of images, could be prepared 
from only this smaller temperature range and then be delivered, for example, to a 
customer. This is a common way of handling temperature images today. The images 
taken by infrared cameras are therefore processed by the image provider and then 
transferred to customers as a view where only a part of the initially captured data are 
shown. 

The customer will typically want to display the IR image sequence, for example, on 
a television or a computer. A problem with this is that this view cannot be further 



processed by the customers. The customer may wish to select other parts of the im- 
age than those selected by the provider of the image. Also, if the temperature span 
for visualizing the background is set to, for example, 20 - 40 degrees Celsius, ex- 
treme events, such as a hot spot (a thermal anomaly such as an overheated electric 
conductor) or an explosion, will saturate in the image thermal range and not be 
shown in the image. 

Object of the Invention 

It is an object of the present invention to enable the display of IR images on a stan- 
dard DV television set, or record it to a standard DV recorder, while enabling the 
retrieval of temperature information from the images. 

Summary of the Invention 

The object is achieved according to the invention by means of an IR camera com- 
prising 

- registration means for registering radiation from an area, or object, 

- conversion means for receiving a signal corresponding to the registered radiation 
from the registration means, 

- DV conversion means for converting the signal to a standard DV format and 
feeding it to a linklayer module, 

- physical layer means for transmitting the converted signal, said IR camera being 
characterized in that it comprises 

- additional data means for providing additional data related to the image proc- 
essing, for inclusion of said additional data in the signal to be transmitted by the 
physical layer means. 

The object is also achieved by a DV processing unit comprising 

- receiving means for receiving a DV stream, said DV stream comprising at least 
one IR image and calibration data 



- sampling means for forwarding each frame to a DV decoder and to an extraction 
means for extraction of calibration data from the DV stream, 

- calculating means arranged to receive the DV stream from the sampling means 
and calibration data from the extraction means and processing the at least one IR 
image on the basis of the calibration data, 

- storage means for receiving the converted image from the calculating means and 
storing it 

With the IR camera according to the invention the user can use a standard DV re- 
corder to record an IR film sequence and be able to play it back at a later time, and 
edit it, using standard software. At the same time, the user has access to calibration 
data, enabling him the selection of data and processing of the film, using dedicated 
software, for example, for measuring the temperature. 

Preferably, the physical layer means of the IR camera, and the receiving means of 
the DV processing unit are adapted to the Fire Wire standard using 32kHz 2-ch 
mode. 

The physical layer means of the IR camera may also be adapted to use the whole 
audio channel instead of just a part of it. This increases the capacity for transmitting 
additional data. 

The additional data means of the IR camera is preferably arranged to provide said 
additional data related to the image processing in the part of the signal normally re- 
served for audio information. In this case, the extraction means of the DV process- 
ing unit is arranged to extract said additional data related to the image processing 
from the part of the signal normally reserved for audio information. 

Said object is also achieved according to the invention by a computer program 
product for use in an IR camera comprising means for outputting a DV signal com- 
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prising at least one image, said computer program when run in the IR camera caus- 
ing the IR camera to perform the following steps: 

- retrieving calibration data stored in a memory means in the IR camera 

- including said calibration data in the DV output signal 

5 

This computer program product is preferably arranged to include said calibration 
data in the part of the DV output signal normally reserved for audio data. 

Said object is also achieved according to the invention by a computer program 
10 product for use in a DV processing unit, which, when executed in the DV process- 
ing unit will cause the following procedure to take place: 

- extraction of calibration data from an incoming DV stream comprising at least 
one IR image, 

- processing of the at least one image on the basis of the calibration data, 
15 - storage of the converted image. 

This computer program product is preferably arranged to extract the calibration data 
from the part of the signal normally reserved for audio information. 

20 

Brief Description of the Drawings 

Figure 1 shows an IR camera according to the invention. 
Figure 2 shows the structure of one Digital Video (DV) frame 21. 
Figure 3 shows one audio block, used according to the invention, assuming that the 
25 32kHz "SD-2ch audio mode" is used. 

Figure 4 shows a typical configuration for providing IR images on DV format. 
Figure 5 shows a block of information structured according to an embodiment of the 
invention. 

Figure 6 shows an apparatus for receiving and processing an incoming DV stream, 
30 according to the invention. 
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Detailed Description of embodiments 

Figure 1 shows an IR camera according to the invention. As is common in IR cam- 
eras, the inventive IR camera comprises an IR detector 1 for registering the radiation 
5 from an area, or object. From the IR detector 1 the registered radiation data is for- 
warded to a calculating unit 3 which performs radiometric calculations on the data, 
i.e. calculates the temperature in each part of the registered area on the basis of the 
registered radiation. 

10 A transforming unit 5 receives the signal from the IR detector 1 and converts it to 

the video standard format CCIR/ITU 601-2/656, also called 4:2:2, to match standard 
television timing output. The converted signal is sent to a display 7 on the IR cam- 
era. The converted signal is also sent to a DV compression unit 9, which converts it 
to a standard compressed DV format and feeds it to a linklayer module 1 1 . 

15 

Typically, but not necessarily, the linklayer module 1 1 also receives audio data from 
an audio unit 13 comprising or connected to a microphone 14, amplifier and a digi- 
tal codec. 

20 The linklayer unit 1 1 combines the DV format data received from the DV compres- 
sion unit 9, including digital synchronization data with audio data received from the 
audio unit 13. From the linklayer module 1 1 the combined signal is transmitted to 
the physical layer, represented by a 1394 PHY unit 15 and from this unit on a Fire- 
wire connection to a DV recorder (not shown). 

25 

To be able to get temperature readouts from IR images, a calculating unit 3 is intro- 
duced according to the invention. The calculating unit 3 is connected to the IR de- 
tector 1 from which it receives additional data acquired by the IR camera electron- 
ics. These additional data are used by the temperature calculating unit 3, together 
30 with the IR images themselves. The additional data comprise calibration constants, 
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registration of background radiation from temperature sensors in the camera and 
possibly other types of data. These data are gathered in containers as the one shown 
in Figure 5. One container is used per image. The containers are sent to the link 
layer module 1 1 which divides each container into fragments that are added to the 
5 DV stream as discussed in connection with Figure 3. In the prior art these additional 
data are not available. 

According to the invention, this data is included in the signal transmitted from the 
1394 PHY unit 15. Normally, the bandwidth is shared between header information, 

10 video information and audio information as explained in connection with Fig. 2. The 
invention uses the fact that the DV standard allows the use of some of the band- 
width normally used for audio data for other types of data if needed. Reducing the 
audio bandwidth of course lowers the quality of the audio data, which is, however, 
still acceptable. At the same time the bandwidth not used for audio data can be used 

15 to transfer additional data such as calibration constants and background radiation 
from temperature sensors in the camera. This additional data can be used by a tem- 
perature calculating unit located, for example, in a processing means in a computer 
on which the IR film sequence is stored, to calculate temperature information which 
may be presented to the user. 

20 

The additional data will be part of the signal adhering to the DV standard, which 
means that any intermediate apparatuses will handle the signal as a standard DV 
signal. The DV signal will be recorded and received by all standard DV equipment. 
When played or received by a computer, a standard DV driver will extract audio 
25 data and video data and display these data to the user. The user can use standard ed- 
iting software tools to cut out interesting parts from the IR film sequence and store 
them. Since DV frames are solid state, i.e. they do not rely on information from pre- 
vious frames and every frame will be filled with IR calibration information, the IR 
DV film sequence will not be destroyed if it is edited at a later time. 
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With a dedicated software driver and dedicated software applications, termperature 
readouts from the DV film sequences can be calculated into temperature data, which 
may be displayed to the user. 

5 As an example, consider the use of the 32 kHz DV audio mode referred to in the DV 
standard as "SD-2ch audio mode". By switching this mode into "Locked mode" to 
synchronize audio and video, 2160 bytes of proprietary data will be available run- 
ning NTSC TV mode. Alternatively, running PAL TV mode 2592 bytes will be 
available. The locked mode ensures that available data bytes will be synchronized, 
10 meaning that the link layer will place them in the currently generated DV video 
frame. 

The Fire Wire standard enables four different audio modes. Apart from the 48kHz 1 
channel mode where all the available bytes are used for sound information, only the 

15 32kHz 2 channel mode supports "locked mode", that is, synchronizing the sound, 
and according to the invention, the IR data, with each image. According to the in- 
vention locked mode is needed to ensure correct interpretation of each image. 
Therefore, at present, the 32kHz 2 channel mode must be used according to the in- 
vention unless a proprietory solution is developed. Alternatively the full audio chan- 

20 nel may be used for additional data. This alternative also requires that a synchroni- 
zation mechanism be added as a part of the calibration data or in some other way. 

When sending DV with embedded IR calibration data over IEEE 13 94 (commonly 
referred to as Fire Wire), one DV frame (one single image) will be divided into 250 
25 valid VDC packages, that is, not counting the additional null package that is sent for 
every 15 th package. A proprietary driver software collects the 250 valid DVC pack- 
ages and assembles them into a full DV frame 21 as shown in Fig. 2. 

Figure 2 shows the structure of one Digital Video (DV) frame 21 . The frame 21 
30 consists of 10 or 12 DIF sequences, although only three DIF sequences 23, 25, 27 
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are shown in Figure 2. Each DIF sequence comprises a header section 29, a subcode 
section 3 1, a VAUX section 33 and an audio and video section 35. Each DIF se- 
quence is divided into 150 DIF blocks 37, 39, 41, 43, holding the sequences 29 - 
35. Each DIF block 37 - 43 has an ID field 45 and a data field 47. The first six DIF 
5 blocks in a DIF sequence comprise header information. The remaining 144 DIF 

blocks are organized in an array of 935 blocks, where the blocks of the first column 
of blocks are normally dedicated to audio data and the rest are dedicated to video. 
According to the invention, however, some of the blocks dedicated to audio data are 
used to carry other types of data. 

10 

Figure 3 shows one audio block, used according to the invention, assuming that the 
32kHz "SD-2ch audio mode" is used. Of the 80 bytes in the block, a first field 51 
comprising three (bytes 0-2) are used for identity information for the block. A sec- 
ond field 53 comprising bytes 3-7 are used for Audio Auxiliary AAUX. Normally 
15 bytes 8-79 would be used for audio data. According to the invention, however, a 

third field 55 comprising bytes 8-55 is used to carry audio data. The remaining field 
57 comprising bytes 56-79 may then be used for other purposes. According to the 
invention, these bytes are used to carry data related to the IR image recording, such 
as calibration data. 

20 

Using the audio block as shown in Figure 3 24 bytes for each audio block can be 
used for other purposes. As discussed in connection with Figure 2, there are 9 audio 
blocks, which means that 216 bytes per DIF sequence can be used for other pur- 
poses. With NTSC TV mode, therefore, 2160 bytes in each frame can be used for 
25 other data such as IR calibration data. The capacity needed for IR calibration data 
would typically be around 1500 bytes, depending on the implementation. 

Fig. 4 shows a typical configuration for providing IR images on DV format, for ex- 
ample, to customers. An IR camera 61 registers thermal data, converts it to a DV 
30 format image and forwards the image to a DV recorder 63. From the DV recorder 
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the image can be forwarded to a computer 65, or displayed on a television screen 67 
The DV recorder may be any type of DV recorder capable of receiving input data 
from an external unit, for example, on a Fire Wire connection. In addition to a stan- 
dard DV recorder it may therefore be, for example a computer or a video camera 
5 having such capabilities, such as a standard hand-held or stationary DV video cam- 
era. 

In the prior art, the forwarded image can only be displayed on the computer or TV 
screen, and only very simple editing can be done, such as selecting a sequence in 
10 time. According to the invention, however, the forwarded image can be subject to 
post processing. In the post-processing for each individual pixel a temperature can 
be calculated, which can in turn be analysed. 

The software used in the computer to analyze the data provided according to the in- 
15 vention must be adapted to the structure of the data provided. This structure can be 
adapted to suit particular needs. Figure 5 shows an example of such a structure. In 
this example, a data container 71 of 1500 bytes comprises linearization parameters 
73, transmission parameters 75, temperature range 77, background radiation 79, 
colorization information 81, camera settings 83 and a time stamp 85. The container 
20 may also comprise other types of data, such as optical parameters (not shown), for 
example, concerning external supplementary optics. This container 71 is bundled 
onto the IRDV stream and can be retrieved by software at the receiving unit. 

Each audio block has a specified position in a frame, which means that identity in- 
25 formation is not needed. To retrieve the data, the program first reads the first 56 
audio DIF blocks of each frame. The program should check the audio source de- 
scription of each frame to ensure that the audio mode is 32kHz SD-2ch. According 
to one embodiment the media stream is controlled by a DCOM server 
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Figure 6 shows the reception and processing of the DV stream, for example, in a 
computer. Most of the units in Fig. 6 perform standard functions and are known to 
the skilled person. An incoming DV stream represented by an arrow 101 is received 
in a receiving unit 103 From the receiving unit 103 the DV stream is fed to a DV 
5 splitter 104 which divides the stream and transmits the audio information, that is, the 
fields 51, 53 and 55 shown in Figure 3, to the sound renderer 105 and the whole DV 
frame to a sample grabber 107. The sample grabber 107 copies the incoming frames 
to an IR calibration data extractor 109 which extracts the calibration data, that is, 
the data shown as 57 in Fig. 3 and forwards it to a calculating unit 111. The calcu- 

10 lating unit 1 1 1 uses the calibration data to set conditions for what should be viewed 
and how, for example, colorization of the IR image. From the sample grabber 107 
the incoming frames are also forwarded to a DV decoder 1 13 as known in the art. 
From the DV decoder 113 the incoming frames are forwarded to a unit introduced 
according to the invention, known as a dedicated IR renderer 115. The IR renderer 

15 115 also receives the conditions from the calculating unit 1 1 1 to use as control in- 
formation when displaying the image. The IR renderer uses the colorization infor- 
mation 79 and other control information to do the following: 

- create a temperature related image to be displayed on the screen, 

- colour the image using an appropriate algorithm for colorization and lineariza- 
20 tion, 

- create numeric analysis information in the form of measurement values, and 

- display analysis results graphically. 

Ideally, every image in the DV stream is converted and stored in a separate file in a 
25 memory means 1 17. 



